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ABSTRACT 

Over a three-year period every school system in 
Maryland received State Department of Education support in putting 
into practice research on effective teaching, using strategies and 
processes from the research on effective staff development and 
planned change. This paper summarizes some of the findings of an 
intensive study of this program, School Improvement Through 
Instructional Process. It discusses factors most strongly influencing 
successful implementation, impact, and institutionalization of four 
resoarch-based innovations: (1) active teaching; (2) mastery 
learning; (3) student team learning; and (4) teaching variables. It 
also explores implications of these findings for the improvement of 
teaching and for effective educational change. (JD) 
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Introduction 



Over a three year peiiod every school system* in Maryland received state 
department support in putting into practice research on effective teaching, 
using strategies and processes from the research on effective staff 
development and planned change. 

This paper summarizes some of the findings of an intensive study of this 
instructional improvement effort.** It discusses factors most strongly 
influencing successful implementation, impact, and institutionalization of 
four research-based innovations. It also explores implications of these 
findings for the improvement of teaching and for effective educational change. 

Program Overviev 

In the three years beg^;ning September 1981, all 24 local education 
agencies (LEAs) in Maryland participated in a School Improvement Through 
Instructional Process (SITIP) program by voluntarily implementing one or more 
of four research-based instructional models: Active Teaching (AT), Mastery 
learning (ML), Student Team Learning (STL), and Teaching Variables (TV). The 
Maryland State Department of Education (MSDE) encouraged the application of 
research on planned change, and supported local implementation by providing 
funds, training, and technical assistance. Evaluation was conducted in order 
to provide relevant information in a timely fashion so that data-based 
decisions could be made about the program. 
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* In Maryland, each county is a school system. The 24 school systems range 
in size from seven to 164 schools, and include large urban LEAs (e.g., 
Baltimore City, Prince George's County) , middle-sized suburban LEAs (e.g., 
Howard County) and rural districts of varying sizes (e.g., Kent and Garrett 
Counties) . 

** Annual and interim reports and related papers were developed. See 
bibliography. 
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SITIP was designed by MSDE as a multi-year program consisting of inter- 
active activities which included cycles of planning, training, and technical 
assistance beginning in late 1980 (see Figure 1). State department support 
(funding and provision of training and technical assistance to LEAs) after the 
first year required commitment of matching funds from the LEAs, and, after the 
third year, was provided only for program expansion. At the end of the third 
year (June 1984) LEAs would institutionalize their model programs, or, if they 
were proven to be ineffective, terminate them, with each district taking 
responsibility for local needs, decisions, and actions. 

Maryland State Department of Education staff engaged in planning, 
designed and conducted a variety of of training activities, and provided 
technical assistance to LEAs. 

• In all local-site interactions, MSDE was a supporter or facilitator of 
instructional improvement, acting on the assumptions that the state 
may influence but cannot control the LEAs, and the immediate responsi- 
bility for instructional change rests with the LEAs. 

• Within MSDE, particularly during the early phases of project design, 
Interactive strategic planning was conducted to establish a clear 
common knowledge br.r.e of local interests, state expectations, and 
relevant research-based alternatives. 

• MSDE sponsored a series of related training activities which were 
carefully designed to link relevant research and exemplary practice.* 
MSDE pre-contracted with LEAs so that expectations and responsibili- 
ties were clear, coached researcher-presenters to meet participants' 
needs, involved LEA teams as presenters to publicize successes, 
conducted "matching" training activities for state staff and faculty 
of colleges and universities involved in pre-service, and provided 
relevant follow-up and on-site coaching (thereby helping LEAs develop 
similar training activities). 



* State-sponsored conferences for SITIP participants included presentations by 
the developers of the four instructional models; by Barak Rosenshine, 
Madeline Hunter, and Jane Stallings on effective instruction; by Robert Bush 
and Bruce Joyce on staff development; and by Karen Seashore Louis and Harold 
Hodgkinson on planned change and school improvement. The MSDE technical 
assistants (TAs) were also strongly influenced by the work of Matthew Miles, 
particularly in the area of program institutionalization. 
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• MSDE helped local implementation by assigning a team of eight part- 
time technical assistants (TAs) . Effective TAs had a sound knowledge 
of instruction, curriculum development, staff development, planned 
change and organizational analysis, and the models to be implemented. 
They were familiar with schools and school systems, particularly those 
to which they were assigned, and they established positive productive 
relationships with those systems, maintaining program integrity in the 
context of local constraints. 

• In order of tfme invested overall, the following tasks were addressed 
by TAs: (1) visiting sites to assist and review local implementation; 
(2) conducting training; (3) developing program activities and 
planning, at the state and local levels; (A) overseeing administration 
and budget; (5) maintaining communication by exchanging ideas within 
the TA system; (6) building knowledge; (7) providing general support 
to local educators; (8) disseminating; (9) selecting and developing 
materials; and (10) evaluating.* During the three year effort, most 
TA time was spent in the second year (a total of about 260 days). 

• From a local perspective, TAs were most useful when they were program 
advocates; provide quality information, training, and assistance 
relevant to local needs to facilitate rich fidelity of implementation; 
and engaged in cross-hierarchical problem-solving that helped to 
clarify program purpose, maintain harmony, and contribute to instruc- 
tional improvement. TAs supported local leadership teams, and 
acknowledged successes. 

The Instructional Models 
Each of the instructional models is described below. These descriptions 
reflect the understanding of Maryland educators, based on training provided by 
the developers in 1980-1981. 

• Active Teaching (AT) is* a system of direct instruction developed by 
Thomas Good and Douglas Grouws at the University of Missouri. 
Originally designed for the teaching of mathematics, AT consists of 
the following components. 

1. Pre-lesson development — concepts and skills from the previous 
night's homework are reviewed, homework is checked and collected, 
and students engage in mental exercises 

2. Lesson development — prerequisite skills and concepts are briefly 
reviewed, and new concepts are introduced via teacher explanation 
and demonstration. 

3. Controlled practice. 



* TA participation in evaluation was primarily information exchange and 

facilitating data collection. Time invested by RBS for evaluation was about 
250 person days per year. 



4. Independent, uninterrupted, individual, successful practice is 
provided in order to increase proficiency in the sk? Is and 
concepts taught. 

5. Homework — the homework that is assigned is re la tae 
concepts developed that day. 

6. Review/maintenance — weekly and end-of-unit reviews help to 
maintain the skills and concepts that have been taught. 

Mastery Learning (ML), developed by Beajamin Bloom (University of 
Chicago) and James Block (University of California), combines 
curriculum alignment and diagnostic/prescriptive instruction with a 
philosophy that all students can succeed. The essential components 
of ML follow. 

1. Developing a scope and sequence of objectives, broken down into 
prerequisite and component skills. 

2. Providing appropriate instruction aligned with the objectives to 
be mastered. 

3. Testing the students' progress in mastering the objectives through 
the use of a formative evaluation measure ("no fault 11 test). 

4. Providing students who have not achieved mastery with additional 
corrective work in the deficient areas specified by the formative 
tests, and providing students who have achieved mastery with 
enrichment activities to reinforce and supplement learning. 

5. Testing final mastery of the objectives with a summative evalua- 
tion measure. 

6. Recording student progress in terms of individual mastery of 
specific objectives. "Mastery 11 is usually defined as 80% of the 
students demonstrating success on at least 80% of the objectives 
in a given unit of instruction. 

Student Team L e arning (STL) techniques use peer tutoring and team 
competition to facilitate student learning. Student Team-Achievement 
Divisions (STAD) and Teams-Games-Tournaments (TGT) were developed by 
Robert Slavin and staff at Johns Hopkins University. Jigsaw was 
started at the University of California at Santa Cruz. The key 
factors of STL are peer interaction, cooperation, and competition. 
STAD is basically team learning; TGT is team learning plus competition 
by ability level; Jigsaw is team learning of specific elements of a 
program, with regrouping for peer teaching across elements. 

Teaching Variables (TV) was developed by David Helms and staff at 
Research for Better Schools (RBS). Two variables found to be strongly 
related to effectiveness of instruction and student achievement were 
identified: "content" and "time." The "content 11 variable encompasses 
the following two factors. 

1. Assessment of prior learning, 

2. Alignment of curriculum objectives and classroom instruction to 
the testing instrument. 



The "time 11 variable improvement cycle involves the following factors. 

1. Measuring student engaged time (SET) via classroom observation. 

2. Comparing SET and opportunity for improvement. 

3. Reviewing and selecting research-based improvement strategies* 

4. Implementing the selected strategies. 

5. Using additional classroom observations to evaluate the effective- 
ness of the strategies in improving SET. 

Evaluation Overview 

The study covered the period December 1980 through June 1984, and 
addressed three areas — institutionalization, impact, implementation, and 
examined both state and local activities. This paper focuses on the latter. 

While Research for Better Schools (RBS) had primary responsibility for 
the SITIP evaluation, the design called for LEA and MSDE involvement. Guide- 
lines were developed and MSDE staff reviewed them with LEA teams. Data were 
collected from MSDE staff and local educators (central office staff, school- 
based administrators, and teachers) representing the 29 projects at 24 LEAs. 
Five general methods of data collection were used: observations, interviews, 
questionnaires, document analyses, and measures of student attitudes and 
achievement. Student achievement was measured in some districts by norm- 
referenced tests (e.g., California Achievement Test), and in others by 
teacher-made criterion-referenced tests. Methodology varied. Data summaries 
submitted to RBS also varied. Therefore, local claims of program impact on 
student academic achievement could not always be empirically verified. 
However, other data collection was extensive, so that the combination of 
information from varied sources collected in varied ways contributed to a 
rich and accurate understanding of the program. Data were analyzed and 
reports were developed by RBS staff, and made available to state staff and 
LEA coordinators so that necessary modifications and improvements could be 
made to the program. 



Local Implementation and Impact 

From the state perspective, successful implementation of SITIP would 
occur if one or more of the models were used by many teachers in many schools 
in all LEAs. The implementation would improve instruction (thereby improving 
students' achievement and attitude toward learning), increase teachers' 
effectiveness, prove to be useful for both elementary and secondary instruc- 
tion in various academic subjects, increase administrators' ability to manage 
planned change, and be carried out in such a way that productive working 
relationships were maintained across role groups. Finally, as state funds 
were gradually withdrawn, MSDE hoped that local SITIP projects would be 
institutionalized, or terminated if instruction had not been improved (with 
that decision based on project results discussed by all role groups). 

The goals of improved knowledge, skills, and attitudes for students and 
teachers were expected by most LEAs. Organizational harmony and administra- 
tive skills were not overt local goals. Also, very few'LEAs were initially 
interested in promoting widespread use or systemic institutionalization, 
although some districts did address those goals after the first year. 

Each LEA was expected to implement a model with "fidelity," to involve 
cross-hierarchical teams in planning and implementation, to send representa- 
tives to state-sponsored training events, to interact constructively with TAs 
and other LEAs implementing a given model, and to provide information relevant 
to program evaluation and student assessment. Each LEA received up to $5,000 
in state funds for Year 1, and up to $3,000 in each subsequent year on 
condition that the local system provided matching funds. 

L£As were free to choose the model (s) most likely to meet local needs, 
and to specify their own implementation strategies and the outcomes they 
expected. Also, each LEA could change plans (e.g., reduce or expand the scope 
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of implementation, terminate a project, or adopt another model), and, in 
making such changes, was encouraged to make data-based decisions (e.g., 
referring to students' test scores, teachers 1 reports, or RBS f studies). If 
the LEAs reduced their workscope or procrastinated, they were offered assist- 
ance to meet their own goals or given the choice of returning state funds for 
that year. 

Implementation Strategies 

During Year 1, it became apparent that staff interest was the most 
influential factor in selection of the model and design of the implementation 
strategy. While work at the school site was strongly influenced by the 
complexity of the model, work across the LEA (how much, how it was shared, how 
workloads shifted among role groups over time) was determined by the strategy. 
In other words, strategies requiring more work across role groups (and leading 
to widespread implementation) were initially selected in LEAs where adminis- 
trators believed that SITIP could address a local priority. More than one 
model was adopted in some LEAs, sometimes with different strategies for each, 
and some models were added or deleted after the first year. When 
implementation was successful, a switch was sometimes made to a more 
work-intensive strategy. When implementation was less successful, a switch 
was sometimes made to a less work-intensive strategy or the project faded 
away . 

The four strategies designed or selected by LEAs are summarized below. 

• District-wide . All schools at a given level (usually elementary) were 
involved, with the selected model used for a given subject all the 
time by participating teachers (at least three per school in the first 
year, all teachers in subsequent years). This strategy required the 
most work from the most people, with central office staff enthusiasm 
and effectiveness important for success. Two projects began with this 
strategy, and by June 1984 a third was also implementing SITIP 
district-wide. All three implemented AT. The largest project 
involved 33 schools. 
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• Pilot/District , One to three schools were involved the first year, 
with strong central office support for school-based activities. 
Evidence of success led to greater administrative involvement and, in 
some cases, use of key teachers as turnkey trainers. This strategy 
was the most feasible, especially for complex models. Five projects 
began with this strategy, and eight were using it by June 1984. The 
largest number of schools involved in a pilot/district LEA was 28. 

• Capacity Building . Training was conducted by the LEA team that 
participated in MSDE institutes. Teachers volunteered to "try" the 
mpdel. There was no formal commitment to follow-up by administrators, 
although where this strategy was effective an administrator did 
"energize" the project. Five projects, all for STL, began with this 
strategy, of which three faded out during the second or third year. 

By June 1984, thers were three capacity building projects (one having 
switched from a lighthouse strategy) with 15 schools involved in the 
largest project. 

• Lighthouse . A single school was involved and no commitment was made 
by central office staff to advocate further use or initiate planning 
or training for other schools. Success was usually broadcast 
informally. This strategy put the greatest burden on school staff. 
There were 20 lighthouse sites initially; 14 by the end of Year 3, 
seven having evolved into pilot/district sites and one into capacity 
building. Two ending as lighthouse sites had begun with other 
strategies. By June 1984, the largest number of schools involved in a 
lighthouse LEA was three. 

For widespread implementation, the lighthouse strategy was least effec- 
tive, but this strategy was successful (from a small-scale perspective) when 
the model matched a principal's priority. Capacity building was least 
effective for maintaining systemic implementation, but did increase teachers' 
knowledge of an alternative instructional model. Overall, the pilot/district 
strategy was most effective, particularly for complex models in large LEAs. 
The district-wide model was successful with less complex models if attention 
was paid to building the commitment of school-based staff. 
Scope, Intensity, and Fidelity of Use 

Influenced by the strategy of implementation chosen and by adminis- 
trators 1 investment of time and interest, the dimensions of scope, intensity, 
and fidelity indicate the nature and extent, of use. 




ERIC 



Scope . Scope of implementation by LEAs in June 1984 is presented in 
Table 1 and is summarized in Table 2. The 23 LEAs in the state receiving 
SITIP funds are listed.* Since several LEAs implemented more than one model, 
there were more than 23 projects. Since each LEA determined allocation of 
SITIP funds, multiple projects within a district were not necessarily equally 
funded, nor given equal attention. The strategies presented relate to those 
employed in Year 3. In several cases the strategies used were different from 
those originally planned. Expansion was influenced by local success (usually 
as perceived by administrators). Reduction (or termination) occurred due to 
minimal impact of SITIP (usually influenced by processes used and environ- 
mental turbulence). All types of schools were involved, including two voca- 
tional-technical centers, ranging from a single school in one LEA to 33 
schools in ar jther. As few as four teachers *re involved in a project to as 
many as 700. The number of students in a project ranged from 113 to 22,594. 

Overall, more than 74,000 students were involved. The 182 schools 
monitored by the study represented about 16% of Maryland's schools. More then 
51% were elementary, usually involving students in grades 3 through 5, Both 
junior /middle and senior high schools were included in the 85 secondary 
schools. About 2,744 teachers used one or more models: additional teachers 
were trained within LEAs and used SITIP ideas at their own discretion. A 
comparison across models indicates that Active Teaching and Mastery Learning 
were the most widely used (impacting about 56% and 36% of SITIP students, 
respectively), and Student Team Learning and Teaching Variables the least 
widely used (impacting about 4% and 5% of SITIP students, respectively). 



One LEA continued implementation alone after the first year, choosing not to 
accept state funds. That LEA is not discussed in this report. 
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Table 1 

Scope of Implementation by LEA: All Models, June 1984 



LEA 


Model 


Strategy 


wot Schools 


Type 


wot Teachers 


for Students 


Allegany 


ML 


LS 


2 


0 


18 


350 


Anne Arundel 


ML 


LS 


1 


H 


5 


300 


Baltimore City 


ML 


PD 


28 


J/M,H 


606 


22,594 


Baltimore County 


ML 


PD 


6 


E 


32 


1,094 


Calvert 


STL 


PD 


3 


E,J/M 


13 


375 




TV 


PD 


3 


J/M 


18 


468+* 


Caroline 


AT 


PD 


7 


E.J/M 


85 


2,695 


Carroll 


ML 


PD 


5 


J/M 


7 


700 


Cecil 


AT 


PD 


25 


E.J/M.H 


700 


13,000 


Charles 


STL 


CB 


15 


E,J/M,H 


116 


650+* 


Dorchester 


STL 


PD 


4 


E 


16 


425 


Frederick 


TV 


LS 


2 


. J.. 

J/M 


14 


350 


Garrett 


AT 


LS 


3 


J/M.H 


20 


1,000 


Harford 


AT 


DW 


33 


E,J/M 


671 


18,650 


Howard 


ML 


PD 


6 


E,J/M 


35 


1,500 


Kent 


TV 


DW 


7 


E,J/M 


52 


1 ,561 


Montgomery 


AT 


LS 


1 


E 


8 


250 




STL 


LS 


1 


J/M 


7 


350 




TV 


LS 


2 


E,J/M 


14 


400 


Queen Anne s 


STL 


CB 


2 


J/M,H 


23 


800 


St. Mary's 


AT 


CB 


7 


E,J/M,H 


62 


1,500** 


Somerset 


AT 


LS 


1 


E 


10 


300 




TV 


LS 


2 




12 


420 


Talbot 


TV 


LS 


1 




13 


250 


Washington 


AT 


LS 




No Data 








ML 


LS 




No Data 








STL 


CB 




No Data 






Wicomico 


AT 


DW 


16 


E 


154 


3,850 


Worcester 


ML 


LS 


1 


E 


8 


240 




STL 


LS 


1 


E 


4 


113 



* At pilot middle school. 
** Includes some duplicates. 



Model ; AT--=Active Teaching 
ML "Mastery Learning 
STL=Student Team Learning 
TV=Teaching Variables 

Type : E-Elementary school 

J/M-Junior high/middle school 
H»High school 
0*0ther 



Strategy : LS=Lighthouse school 
PD=Pilot district 
DW-District wide 
CB=*Capacity building 
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Table 2 



Summary of Scope of Implementation: All Moc'els, June 1984 





Pro. 


ects 


Schools 


Teachers 


MOG61 




n 


Ma* 


So 




^ 


Active Teaching 






E 65 
S 28 










9 


31 


93 


C 1 

51 


MIA 

1 / 1U 




Mastery Learning 






E 10 
S 37 
0 2 










8 


27 






"7 11 
/ll 


ZD 


Student Team Learning 


6 


20 


E 15 

n 11 

S 1 1 
26 


14 


200 


7 


Teaching variables 


6 


20 


E 6 
S 10 
0 1 
17 


9 


123 


5 


Total 






E 96 
S 86 
0 3 










29 


100 


185 


100 


2744 


100 



* Three schools (two elementary and one secondary) are implementing two 
models. 



** Eighteen teachers are implementing two models. 

Schools : E ■ Elementary 
S * Secondary 
0 ■ Other 

N ote : No data available for Washington County (Active Teaching and 
Mastery Learning). 
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While the relative simplicity of AT facilitated its expansion, complexity was 
not a deterrent: Mastery Learning was used by 27% of the SITIP schools. 

Intensity . The average number of years that teachers were involved in 
SITIP was 1.6 for AT, 1.8 for ML, and 1.9 for STL and TV (while teachers 
involved from the beginning participated for three years, mean times were 
reduced by the large number of teachers in expansion sites). During the 
1983-84 school year, teachers used STL for an average of five months, ML and 
TV for seven months, and AT for close to nine months. AT and ML teachers used 
the models for a larger percentage of their in-class tine (an average of 51% 
and 43%, respectively) than teachers using TV (36%) and STL (19%). Consistent 
use facilitated instructional gain. 

Fidelity . Each model required the implementation of certain components. 
More teachers (91%) implemented all critical components of the AT model than 
did the implementers of the other three models (ML — 62%, STL — 59%, TV — 
462; "time 11 , 18% "content"). With the exception of ML, which was second only 
to TV in complexity, the more complex the model was to implement, the less the 
degree of fidelity. The degree of fidelity was also related to the extent of 
administrator "press 11 for fidelity of implementation. In those LEAs where 
administrators encouraged and expected to see fidelity of implementation, more 
teachers implemented all components of the model. Such "press," plus support 
provided in Years 2 and 3, probably facilitated the fidelity and intensity of 
ML (overcoming the uncertain implementation experienced in some LEAs in 
Year 1). 

Adm in i s t r a t i ve in v e s tme n t . The average amount of time invested by a 
SITIP administrator during Year 3 was 21.05 days, with means ranging from 
16,45 days for AT to 26.65 days for ML. Activity areas, in order ol priority 
allocation of time, included: (1) inservice, (2) general support, (3) 
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administration/ communication, (A) monitor ing/evaluat ion, and (5) dissemina- 
tion/expansion. While inservice and support were top priorities for three 
models, top priorities for TV were administration/communication, and 
monitoring/evaluation. Time investments and priorities were influenced by the 
nature of the model and the scope of implementation. Results suggest that 
success (in terms of instructional gain or institutionalization) is 
facilitated by administrative involvement in inservice and general support. 

In general, the quality of implementation varied. While there were some 
exemplary sites for all models, there were others where fidelity was low, or 
application was sporadic. Poor implementation was characterized as infrequent 

i 

use of a model, pro forma application of parts of a model, lack of actual 
change in classroom behavior, or isolated teachers carrying out an adaptation 
as best they could. Excellent implementation was characterized by definable 
changes in classroom behavior, increased student time-on-task directly linked 
with aligned curriculum and quality instruction, use of the model regularly 
and/or for a complete unit or course; and data-based decision-making. 
In the sites where better implementation occurred, administrators were well 
informed, supportive, and expressed clear expectations of fidelity and 
intensity. 

Roles and Responsibilities 

The SITIP design encouraged participatory decision-making and involvement 
of all three instructional role groups in an LEA. By the end of Year 2, it 
was apparent that: (1) teachers involved in MSDE training activities 
sometimes became instructional leaders, and all teachers involved in SITIP 
needed time to develop materials, and support and assistance in implementa- 
tion; (2) school-based administrators Involved in MSDE training activities 
were more committed than those trained by LEAs, and all needed to support 
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teachers' efforts for success; and (3) central office staff, after MSDE 
training, determined their roles by the extent to which a model met local 
priorities, contributing most effort through inservice or general support, but 
contributing relatively little (e.g., only administration) when a lighthouse 
strategy was used or a model was perceived as more teacher-centered. Commit- 
ment of any role group was influenced by the extent to which individuals 
believed they had been given some area of choice. 

In Year 3, participants, particularly the LEA teams that initiated local 
projects, were aware of each other's relative success and the processes and 
factors that inhibited or facilitated that success. They were advised by MSDE 
to consolidate successes, make appropriate revisions, and make data-based 
decisions to terminate or institutionalize as state funds were withdrawn. 
Particular attention was to be paid to interactive support and leadership • 

Interactive support . Support among LEA participants included exchanging 
information and materials; providing training, coaching, and trouble-shooting; 
managing logistics; and recognizing successes. Support from MSDE and 
developers consisted primarily of training, technical assistance, networking, 
and trouble-shooting. As in previous years, the effects of visibility 
(frequency and accessibility of interactions) were apparent, with higher 
ratings awarded to role groups more visible to teachers. When several role 
groups were fairly equally visible, expertise and affective and logistical 
support influenced ratings. In general, developers, who interacted very 
little with local educators, received the lowest ratings, although STL 
developers, who were the most visible, were rated slightly lower than ML 
developers. Overall, MSDE staff received the next lowest ratings (all above 
average) with the expertise of ML TAs and the locally-responsive networking 
style of the STL TAs being well-perceived. While school-based administrators 
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were rated somewhat higher than central office staff overall, the range for 
the latter group across models was much wider than for the former. This 
indicated that school-based administrators played similar roles, regardless of 
the model, but roles played by central office staff differed by model in 
visibility and demonstration of expertise, and in affective and logistical 
support. Support by teachers was rated most highly overall, with lower 
ratings for AT (which was the least complex model), and TV (which made the 
least demands on non-observing teachers) , and higher ratings for STL (which 
was teacher-led in many districts), and ML (which was the most complex model 
and made considerable demands on teachers) . 

In comparison to Year 2, overall ratings for each role group were 
slightly lower, suggesting the diminution of energy which might be expected as 
institutionalization occurs. Slight increases were awarded to teachers for 
STL and TV, and to central office staff for ML, which related to extra invest- 
ments of effort which they made. Below average ratings awarded to TV central 
office staff were related to the fact that the role group was involved in only 
two of the projects. 

Administrative leadership . Affective and logistical leadership behaviors 
are presented in Table 3 together with ratings assigned to central office 
staff and school-based administrators for each of the models. Overall ratings 
for central office staff ranged from 2.49 (press for fidelity) to 3.99 (demon- 
strate commitment). Overall ratings for school-based administrators ranged 
from 2.63 (press for fidelity) to 4.23 (demonstrate commitment). For both 
role groups, affective behaviors were more evident than logistical behaviors, 
and organizational process behaviors were more evident than those related to 
"press/ With the exceptions of central office staffs' data-based decision- 
making, and school administrators' press for fidelity and intensity, all 
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Table 3 



Administrative Leadership Behaviors: All Models, 1983-84 











Mean 


Rat im 


ks Assigned ~~| 


Behaviors 


To 


centre 


il office staff 


To school administrators 




AT 
N-94 


ML 
N-81 


STL 
N-43 


TV 
N«53 


all 


AT 


ML 

KT.QQ 


STL 


TV 
N"60 


all 
N-291 


Affect ive 






















Demons t m t pAmw A fmanf 


H . UO 


1 .32 


3.93 


3.35 


3.99 


U 55 


L 1 7 




J . 95 


4.23 


Provide support 


3.45 


4.06 


3.74 


3.11 


3.61 


4.12 


3.98 


3.82 


3.80 


3.97 


Logistical 






















Press for f-Mpl-ft-v 


9 77 


z.t>4 


2.16 


2.04 


2.49 


3.02 


2.72 


2.10 


2.23 


2.63 


Press for intensity 


2.95 


2.69 


2.33 


2.04 


2.59 


3.28 


2.80 


2.30 


2.32 


2.79 


PrOV*fHp QQa , l afan/iA 

* ^wv auc aaslotflnCG 


J . 39 


4 . 16 


3.86 


3.00 


3.61 


3.80 


4.09 


3.83 


3.62 


3.85 


Coordinate LEA 
communication 


2.97 


3.76 


3.59 


2.69 


3.25 


3.19 


3.40 


3.40 


3.00 


3.25 


Coordinate school 
communication 


2.81 


3.61 


3.37 


2.68 


3.11 


3.57 


3.55 


3.40 


3.30 


3.48 


Implement data-based 
decision -making 


2.76 


4.04 


3.30 


3.88 


3.47 


3.09 


3.71 


3.24 


3.17 


3.31 



Scale ranges from 1.00 (not at all) to 5.00 (to a very large extent), 
teaching; ML-mastery learning; STL=student team learning; TV-teaching variables. 
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ratings for leadership behaviors were lower for TV than for other models. For 
all models except ML, ratings were higher for school administrators than for 
central office staff. 

In all cases, affective leadership behaviors were above average. 

Logistical leadership behaviors relating to the organizational processes 
of provision of assistance, coordination of communication, and implementation 
of data-based decision-making, were above average with exceptions for the last 
three behaviors for central office staff in AT, and for communication 
behaviors for TV. Press for fidelity and intensity by school administrators 
were above average only for AT. 

An analysis of variance showed significant differences between the four 
models on central office support (see Table 4). TV had the lowest, and ML had 
the highest mean on this index. There also were significant differences 
between the four implementation strategies on this index (see Table 4). The 
lighthouse school strategy had the lowest mean, and the pilot/district and 
district-wide strategies had the highest means on central office support. 
There were no significant differences between models or strategies on school 
administrator support. 

"Press" indicated administrative expectations of fidelity and intensity, 
without which teachers could assume that it was acceptable for them to make 
little or no change. Low administrative press was related to low success and 
potential project decline. 

In several LEAs, leadership was undertaken by teachers (with administra- 
tive support). In s.ome cases, t« ams of key teachers conducted training and 
coaching, and, for TV, conducted classroom observations. In other cases, 
individual teachers ran the project, usually in a single school, but in one 
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Table 4 



ANOVA Results for Central Office Support 



Factor 


N 


X 


F 


df 


P 


1. Model 


228 


3.30 


9.19 


3/224 


.001 


AT 


77 


3.12 








ML 


65 


3.77 


STL 


37 


3.46 


TV 


49 


2.84 


2. Strategy 


228 


3.30 


10.14 


3/224 


.001 


Lighthouse school 


87 


2.86 








Capacity building 


21 


3.19 


Pilot district 


91 


3.64 


District-wide 


29 


3.64 



AT » Active Teaching; ML » Mastery Learning; 

STL = Student Team Learning; TV - Teaching Variables 
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case across the LEA (with release time to do so). The strongest leadership 
behaviors of teacher leaders were provision of assistance (when they had been 
trained at MSDE events) , and support (when they believed in the model and had 
release time to help their colleagues). The weakest behaviors were coordina- 
tion among schools (when they had insufficient release time and little 
influence on other schools), and press for fidelity (when they had low 
expertise in the model or in influencing others). In order to be effective, 
teacher leaders had to have real expertise in the model and strong administra- 
tive support. 

Overall interactive support and leadership were good, and for most 
projects improvements were made over the three years. However, there were 
problems if reassignments resulted in leaders who lacked expertise or commit- 
ment,* if central office staff functioned only as administrators, if 
principals had priorities addressed by activities very different from SITIP, 
or if teachers were expected to do most of the work with little support. In 
contrast, where project teams remained stable and project management tasks 
were shared, where leadership behaviors were above average, and where 
expertise in the model helped achieve an existing priority, implementation was 
smoother, impact was more evident, and institutionalization more probable. 



Outcomes 



Institutionalization of successful projects was the desirable outcome for 
Year 3, and indicators were identified to determine the extent to which that 
was occurring. In addition, impact on students and teachers was assessed to 
determine instructional gain. 



* Multiple reassignments — several key staff changed in one year, or prelect 
leadership changed each year - resulted in loss of expertise and momentum, 
contributing to project decline. 
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Students. As indicated in Table 5, students enjoyed SITIP classes (with STL 
being most popular), increased their achievement (most obviously in ML), 
retained more of what was taught (most obviously for AT), took somewhat more 
responsibility for their own learning (more so for AT), and, in general, 
behaved a little better (more so for AT). Empirical data - summaries of 
results of standardized tests and analyses of student progress comparing SITIP 
classes and non-SITIP classes - supported educators' perceptions that student 
achievement was significantly higher when AT or ML was implemented, 
particularly in mathematics. No standardized test data were provided for STL 
or TV. 

Teachers. Teachers* knowledge of effective teaching and skill in 
instruction improved. 

Impact on teachers and students combined was defined as instructional 
gain. A one-way analysis of variance showed significant differences between 
the four models on instructional gain (see Table 6). TV differed from the 
other models on this index. Results showed that the mean for TV: (1) had the 
largest deviation from the total group mean, and (2) was the only model mean 
lower than the group mean. 

There wore also significant differences between elementary and secondary 
schools on instructional gain. The mean on this index was significantly lower 
for secondary schools than for elementary schools. (This may have been 
influenced by the fact that SITIP models were more often used in the latter 
for basic skills. ) 

Schools. As indicated in Table 5, educators agreed that the SITIP models 
worked in the classroom (with AT most strongly affirmed), and that they were 
worth the work they took (with strongest agreement apparent for AT). 



9 

ERIC 



2i 24 



Table 5 

Instructional Impact as Perceived by Survey Respondents: All Models, 1983-84 





Models 


Impact on Instruction 


AT 
N a 124 


ML 
N-97 


STL 
N«54 


TV 
N«72 


Total 
N-347 


instructional value 












Works in the classroom. 

Is worth the work it takes. 


4.50 
4.29 


4.35 
4.05 


4.33 
3.96 


4.14 
3.82 


4.36 
4.14 


Impact on Teachers 












Teachers enjoy it. 

Teachers have increased knowledge. 

Teachers have increased skills. 


4.06 
4.01 
4.05 


4.02 
4.24 
4.19 


4.02 
3.96 
3.77 


3.60 
3.94 
3.92 


3.94 
4.02 
4.02 


Impact on Students 












Students enjoy it. 

Students' achievement has increased. 
Students are learning/retaining more. 
Students' general behavior is better. 
Students are taking more responsi- 
bility for their own learning. 


3.99 
3.87 
3.89 
3.70 

3.56 


4.13 
4.04 
3.79 
3.55 

3.73 


4.47 
3.69 
3.63 
3.60 

3.77 


3.46 
3.47 
3.38 
3.58 

3.31 


4.00 
3.81 
3.72 
3.62 

3.59 



Mean ratings range from 1.00 (strongly disagree) to 5.00 (strongly agree). 
" ^achVv^ief " MaStery Le " nin *' ^ ' »— ' Tea* Learning; 
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Table 6 

ANOVA Results for Instructional Gain: All Models 



Fac tor 


N 
H 


A 


F 


df 


P 


1 . Model 


335 


3.93 


4.65 


3/331 


.003 


AT 


124 


4.00 


8.85 


1/318 


.003 


ML 


93 


4.01 


STL 


51 


3.96 


TV 


67 


3.67 


2. School Type 


320 


3.92 


Elementary 


160 


4.03 








Secondary 


160 


3.81 



AT * Active Teaching; ML * Mastery Learning; STL - Student Team 
Learning; TV * Teaching Variables 



School organizational outcomes were fairly good, although local 
"ownership 11 was only moderate (see Table 7). Policy outcomes (Table 8) 
indicated that shared management and data-based decision-making were more 
apparent for SITIP than for other programs. Procedural outcomes (Table 9) 
indicated that modification of inservice and staff assignments was occurring 
to a greater extent than allocation of resources and use of local funds. 
However, other data sources indicated that most LEAs made significant in-kind 
contributions . 

The three sets of outcomes — organizational, policy, and procedural — 
made up the indicators for school institutionalization. An analysis of 
variance showed that there were no significant differences between models or 
strategies for school institutionalization. 

A multiple regression analysis was conducted to determine which indices 
(instructional gain, central office support, or school administrator support, 
or fidelity) were the best predictors of school institutionalization (see 
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Table 7 



Organizational Outcomes: 


All Models, 1983 


-84 


Outcomes 


School 
N-310 


System 
N-55 


Cognitive 






Status of SITIP established. 


3.76 


3.83 


Close to IOjZ of teachers asked 
to participate do so regularly. 


3.61 


3.35 


Affective 






Local educators feel "ownership" 
of SITIP. 


3,27 


3.24 

V • mm ~ 


There is harmony between teachers 
and school-based administrators 
about SITIP. 


3.87 


3.78 


There is harmony between school- 
based and central office staff 
about SITIP. 


3,63 


3.95 



Scale ranges from 1.00 (not at all) to 5.00 (to a very large extent). 
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Table 8 

Policy Outcomes: All Models, 1983-84 



Policies 


School 


Sys 


tem 




N 


Mean 


N 


Mean 


Management is shared. 


76 


3.96 


c 8 


3.50 


Decision!, are data based. 


313 


3.49 


57 


3.51 j 



Scale ranges from 1.00 (not at all) to 5.00 (to a very large extent). 



Table 9 

Procedural Outcomes: All Models, 1983-84 



Procedures 


School 


System 




N 


Mean 


N 


Mean 


Inservice modified to support SITIP. 


309 


3.75 


64 


3.61 


Staff assignments and accountabili- 
ties modified. 


317 


3.73 


57 


3.35 


Resources allocated annually. 


315 


3.50 


57 


3.68 


Local funds used. 


70 


3.07 


51 


3.15 



Scale ranges from 1.00 (not at all) to 5.00 (to a very large extent). 
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Table 10).* Together, the four indices explained approximately 48% of the 
variance in school institutionalization, which was significant at the .05 
level. The strongest predictor of school institutionalization was school 
administrator support, followed by instructional gain (see Table 10). School 
institutionalization was also strongly correlated with central office support 
(see Table 11). 

Table 10 



Multiple Regression Results for School Institutionalization 



Index 


B 


F 


School administrative support 


.4928 


27.320* 


Instructional gain 


.2365 


5.614* 


Central office support 


.1183 


1.364 


Fidelity 


.0732 


.563 



R 2 = .47884 
Overall F = 17.538* 
N = 73 
*p = less than .05 



School system . Institutionalization indicators of outcomes relating to 
organization, policy, and procedures at the system level are presented in 
Tables 7, 8, and 9, with ratings given only by administrative and supervisory 
staff. In comparison to the school level, ratings assigned at the system 
level for the status of SITIP and for school and system harmony were somewhat 
higher. Also slightly higher at the system level were data-based decision- 
making, allocation of resources, and use of local funds. Of some concern were 
the extent of teacher participation, local ownership, modification of staff 



* Seventy-three teachers responded to all five indices and were included 
in the calculation. 
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assignments, and use of local funds (although the ratings on the latter were 
somewhat misleading given the considerable investments of in-kind contribu- 
tions). It should be noted that system-level outcomes were less important in 
LEAs focusing on a lighthouse school approach. However, from an overall 
cost-effective perspective, higher ratings were desirable since they indicated 
greater likelihood of district-wide institutionalization. 

An analysis of variance showed significant differences between the four 
implementation strategies on system institutionalization (see Table 12). 
Results showed that the mean for the lighthouse school strategy: (1) had the 
largest deviation from the total group mean, and (2) was the only strategy 
mean lower than the group mean. 

There were no significant differences between the models on system 
institutionalization. 

Correlation among the five indices showed strong direct relationships 
(r - .50) between system institutionalization and two indices — central 
office support and school institutionalization (see Table 11). 

Table 11 

Intercorrelation Among the Five Indices: All Models 



Index 


1 


2 


3 


4 


5 


L. 


Instructional Gain 




.44 


.34 


.23 


.26 


2. 


System Institutionalization 






.67 


.80 


.42 


3. 


School Institutionalization 








.51 


.68 


4. 


Central Office Support 










.45 


5. 


School Administrator Support 













Note: The number of cases upon which the correlations were calculated varied. 
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Table 12 

ANOVA Results for System Institutionalization 



Factor 


N 


X 


F 


df 


P 


Strategy 


48 


3.60 


10.76 


3/44 


.001 


Lighthouse school 


12 


2.51 








Capacity building 


4 


3.73 








Pilot/district 


24 


4.01 








District-wide 


8 


3.95 









Summary and Conclusions 

Rather than repeat preceding discussions, this section attempts to answer 

questions most often posed by researchers, policy makers, and practitioners 

who are interested in large scale instructional improvement. Most such 

questions are contained in the overall question: 

• If the "bottom line" is instructional gain, and if chat is 

accomplished by bringing about long-lasting, worthwhile changes in 
teachers' behavior, what are the processes and content uo be applied 
by large systems such as state departments or large school districts? 

Since STTIP was informed by the research on classroom and school effectiveness 

and planned change, the authors of this report believe that the processes, 

findings, and conclusions are generalizable , and may well prove to be useful 

to those involved in similar projects elsewhere. 

If an instructional improvement program is initiated, what indices should be 
monitored to gauge the "health" of the program and the probability of eventual 
institutionalization ?* 

• Indicators of program fidelity and intensity include: 

1. The extent to which participating educators carr, out critical 
components of the program regularly and/or continuously. 

• Indicators of instructional gain include: 

1. Impact on teachers: increase in knowledge and skills, positive 
attitude to the program. 
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2. Impact on students: increase in achievement, learning, and 

retention; improvement in general behavior and the extent to which 
they take responsibility for their own learning; a positive 
attitude toward the program. 

• Indicators of administrative support include : ** 

1. Affective behaviors that... 

(a) demonstrate commitment and belief in the program's value 

(b) provide support by demonstrating interest and recognizing 
teacher success. 

2. Such logistical behaviors as... 

(a) a positive "press" for fidelity, monitoring implementation, 
and expecting a given level of use of the program 

(b) a positive "press" for intensity, monitoring implementation* 
and helping to ensure that at least three teachers in each 
participating school uee the program regularly 

(c) providing assistance by coordinating, training, responding to 
requests, and providing resources 

(d) coordinating communication across hierarchical levels for 
program review and improvement 

(e) implementing data-based decision-making. 



• Indicators of institutionalization include: 
1. Organizational outcomes 

(a) cognitive: the status of the program is commonly understood, 
clearly stated, and close to 100% of teachers asked to 
participate do so regularly 

(b) affective: local educators feel "ownership" of the program; 
there is harmony between teachers and school-based adminis- 
trators about the program; and th^re is harmony between 
school-based staff and central office staff about the program. 



* Tasks that are specifically administrative (e,g., budget) take minimal 
time, and are subsumed under 2c of administrative support. 

** The authors acknowledge the influence of Matt Miles in determining these 
indicators. See: Miles, M,B. Unraveling the mystery of institutionaliza- 
tion. Educational Leadership . November 1983, 14-19. 
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2. Policy outcomes 

(a) management (leadership, advocacy, decision-making) is shared, 
not reliant on a single administrator 

(b) effectiveness is assessed and data are used in decision- 
making. 

3. Procedural outcomes 

(a) inservice is modified to support the program 

(b) staff are assigned and accountabilities are modified 

(c) resources (time, materials) are allocated annually 

(d) local funds are used. 

• The strongest predictors of school institutionalization are support 
from school-based administrators and instructional gain. This 
indicates that the program selected has to be one that really makes a 
difference :.n the classroom, and is sufficiently linked to the 
principal's p-.iorities to influence administrative investment in 
affective and logistical leadership behaviors. 

• District-wide institutionalization is strongly correlated with central 
office support and school-level institutionalization. The lighthouse 
school implementation strategy does not facilitate district-wide 
institutionalization. However, central office support is more evident 
when pilot/district or district-wide approaches are used, and in 
programs with high probability of instructional gain. 

• If the program selected has proven its value elsewhere, but results in 
little or no instructional gain at a new site, the fidelity and 
intensity of use should be assessed. If both are high but apparent 
for only a few isolated teachers, administrative support needs to be 
improved and organizational, policy, and procedural outcomes assessed 
and modified if institutionalization is to occur. 



If the innovation is directly related to a local priority, how can an LEA 
design i mplementation to ensure that program benefits are greater than 
investments (of staff, time, and funds) ? 

• A cross-hierarchical team should be formed that takes responsibility 
for planning, making decisions, and modifying activities by using 
information about the relative effectiveness of the program. This 
team (plus other representatives of role groups) should have a 
thorough understanding of the innovation so that plans are realistic 
and policy and practice are interactive. 
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• A pilot/district or district-wide strategy can be used. In both 
cases, the goal is for all program eligible teachers to be 
implementing the model regularly by the end of the third year, and a 
process of Incremental involvement is used. The pilot/district 
strategy begins by focusing on a very few schools, and expands by 
school, beginning in each school with the principal's support and an 
active team of volunteer teachers. The district-wide strategy begins 
by focusing on teachers (from all schools) with responsibility for a 
given subject area and grade level(s), and expands by grade level (and 
sometimes also by subject area). In both cases, awareness training 
should be conducted for all administrative and supervisory staff 
before teachers are trained. "First Wave" participants should be 
volunteers to the extent feasible. 

• Attention should be paid to the indicators of institutionalization, 
particularly those relating to organizational outcomes and administra- 
tive support. 

• Participating teachers should be given release time, and school teams 
should have common planning time in their first year of implementation, 
with more time available if curriculum materials are to be developed. 

• Classroom instruction should not begin until teachers are prepared to 
teach a complete unit or course. Stops and starts, sporadic implemen- 
tation, and low fidelity should be discouraged by team leaders 
providing relevant coaching or support so that participating teachers 
can experience success. 

• Assessment of implementation processes and instructional gain should 
be on-going to inform decisions — replicating successes, and dealing 
with problems as soon as they are identified. 

How can training activities be most useful in improving classroom instruction ? 

• The overall design for training should include various kinds of 
activities for various audiences. In general, trainers should 
recognize that the more intense the intended outcome, the greater the 
frequency of trainer/trainee interaction, and the smaller the ratio of 
trainers to trainees. Also, the less intense the outcome (e.g., 
awareness), the less investment is likely from participants.* 

• While participants enjoy the less intense kinds of training 
activities, they value and are more likely to be influenced by the 
more intense activities (especially when content is highly relevant). 



* See Joyce, B.R. , 4 Showers, B. Power in staff development through research 
on training . Alexandria, Va.: ASCD, 1984. Components of the model include: 
rationale and theory building (for awareness), demonstration and modeling 
(for conceptualization), practice and feedback (for skill development), on- 
site coaching (for application or horizontal transfer), and integrated 
learning (for executive control or vertical transfer). 
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As the project progresses, training activities should increase in 
intensity, and when participants have "executive control" of the 
model/innovation, training should stop. 

• Training should be directly related to participants 1 need to know, 
building on existing knowledge, and addressing current tasks and 
interests. Pre-contracting clarifies mutual expectations of 
participants' responsibilities for action following training. 

• Training should be designed to be transferred, using a trainer of 
trainers approach, or expecting district and school teams to follow 
through on the more intense activities. 

• Team training, with common activities for everyone as well as activi- 
ties for specific roles and for groups with varying levels of 
expertise, should address both program content and implementation 
processes. 

• Training should be conducted by outside "experts," such as program 
developers, and by state and local instructional leaders, with each 
event including trainers from several role groups. ("Outsiders" 
should be carefully coached about trainees' interests.) 

• Teachers who conduct training usually are most successful when they 
form trainer teams of two or three people. Individual teacher- 
trainers need strong support from administrative staff. 

• The final components of training (on-site coaching and integrated 
learning) can be conducted by three-person teacher teams in each 
school which are supported by a district-wide network. 

• While the integrity of the knowledge base (program fidelity and 
completeness) should be maintained, trainers should be flexible in 
the delivery of training, modifying methods to meet participants' 
needs (e.g., using different approaches for different schools). 



If the innovation(s) match local priorities, and the research on planned 

change is applied to facilitate implementation, what is the likelihood of 

local institutionalization of an externally-initiated program ? 

• The likelihood of some projects being institutionalized Is increased 
by the use of the research on planned change, but schools are Impacted 
by the constant changes of society. Schools and school systems cannot 
accurately predict or control the social pressures or environmental 
turbulence that can change priorities or undermine programs. However, 
they can select innovations most likely to address the basic business 
of schooling — effective instruction — in which they naturally 
invest their own efforts and for which external support may be a 
welcome temporary addition. 
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• In lighthouse schools, institutionalization is probable if the 
principal's priorities are addressed, there are no conflicting 
innovations, and at least three teachers advocate and implement the 
innovation regularly, 

• Institutionalization is unlikely if funds are used primarily for 
training (a capacity-building approach) with voluntary application by 
trainees, unless school teams pre-contract and those teams receive 
fo". low-up assistance in the context of an administrative "press" for 
imp 1 erne n t a t i on . 

• Institutionalization is likely where the indicators (described 
earlier) are attended to from the beginning of the program, where 
materials development is essentially completed by the middle of the 
second year, and where total local ownership is expected by the end of 
the third year. 



If instructional gain is the intended outcome, what kind of innovation is best ? 

t Instructional gain is defined as (1) increase in teachers 1 knowledge 
and skill in effective instruction and a positive attitude toward the 
program, and (2) increased student achievement and acceptance of 
responsibility for their own learning, and a positive attitude towards 
the program, 

• Since instructional gain is such a comprehensive construct, it is 

unlikely to be achieved without careful implementation and planning, a 
reasonable scope and intensity of use, and application of an innova- 
tion designed to achieve such an outcome. The four models used in 
STTIP were so designed, and careful attention was paid to implementa- 
tion processes. 

Assuming appropriate implementation and good fidelity, greatest 
instructional gain is likely if Active Teaching or Mastery Learning 
are used, with best results in elementary mathematics, or for 
structured academic subjects in secondary schools. Student Team 
Learning results in somewhat less instructional gain since educators 
tend to use it sporadically. Least gain is likely for Teaching 
Variables since educators tend to use it for assessment rather than 
from improvement. 
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